User Interface for the Application of Image Effects to Images

ABSTRACT

An apparatus comprising: at least one processor; and at least one memory including computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following: in response to user input selecting a selectable marker ( 1720, 1725, 1730, 1820, 1825, 1830, 1940, 2020 ), the selectable marker marking the position of at least one identified feature ( 1705, 1710, 1715, 1810, 1815, 1910, 1915, 2005 ) in an image which is hideable from view using a region of the image surrounding the identified feature, hide or show the identified feature.

TECHNICAL FIELD

The present disclosure relates to the field of user interfaces and/orimage processing, associated methods, computer programs and apparatus.Certain disclosed aspects/examples relate to portable electronicdevices, in particular, so-called hand-portable electronic devices whichmay be hand-held in use (although they may be placed in a cradle inuse). Such hand-portable electronic devices include so-called PersonalDigital Assistants (PDAs), mobile telephones, smartphones and othersmart devices, and tablet PCs.

The portable electronic devices/apparatus according to one or moredisclosed aspects/examples may provide one or more audio/text/videocommunication functions (e.g. tele-communication, video-communication,and/or text transmission (Short Message Service (SMS)/Multimedia MessageService (MMS)/emailing) functions), interactive/non-interactive viewingfunctions (e.g. web-browsing, navigation, TV/program viewing functions),music recording/playing functions (e.g. MP3 or other format and/or(FM/AM) radio broadcast recording/playing), downloading/sending of datafunctions, image capture function (e.g. using a (e.g. in-built) digitalcamera), and gaming functions.

BACKGROUND

The application of image effects to images can assist a user to producecreative images. Images can be manipulated and effects can be applied ona multitude of ways with a multitude of settings. Thus, the applicationof image effects to an image and/or the manipulation of an image can betime consuming for a user. Further, image effects can take anon-imperceptible time to apply to an image and therefore a user mayhave to apply and then undo changes to an image several times until theyfind the image effect and effect parameters that they are happy with orthat suits the image.

The listing or discussion of a prior-published document or anybackground in this specification should not necessarily be taken as anacknowledgement that the document or background is part of the state ofthe art or is common general knowledge. One or more aspects/examples ofthe present disclosure may or may not address one or more of thebackground issues.

SUMMARY

According to an aspect of the invention, we provide an apparatuscomprising: at least one processor; and at least one memory includingcomputer program code, the at least one memory and the computer programcode configured to, with the at least one processor, cause the apparatusto perform at least the following:

-   -   in response to user input selecting a selectable marker, the        selectable marker marking the position of at least one        identified feature in an image which is hideable from view using        a region of the image surrounding the identified feature, hide        or show the identified feature.

The selectable marker may comprise a ring or circular ring thatsurrounds the at least one identified feature.

When a plurality of features are identified, each feature may be markedby a respective selectable marker comprising a ring or circular ringthat surrounds the respective feature.

The or each selectable markers may include a plus symbol when theidentified feature is hidden in the image and includes a minus symbolwhen the identified feature is shown in the image.

The plus symbol and minus symbol may be actuatable.

The plus symbol for a marker may allow for the hidden feature to beshown and the minus symbol for a feature may allow for the feature to behidden.

The apparatus may be configured to initially provide for display of theimage without any of the identified features hidden and the selectablemarkers are initially configured to allow for the hiding of a respectivefeature in response to said user input selection.

The apparatus may be configured, when two or more features areidentified, associate the two or more identified features such that theyare treated as a single feature by determining if said two or moreidentified features satisfy a predefined clustering criterion.

The apparatus may be configured such that said association allows thetwo or more associated features to be shown or hidden together inresponse to said user input selection using a single marker marking theposition of the associated features.

The mark applied to mark the position of associated features maycomprise a ring that surrounds the associated features.

The predefined clustering criterion may be satisfied if the region ofthe image surrounding the one of the identified features overlaps withthe region of the image surrounding the other of the identifiedfeatures.

The predefined clustering criterion may be satisfied if the two or moreidentified features are within a predetermined distance of one another.

The predefined clustering criterion may be satisfied if the regions ofthe image surrounding the two or more identified features are of a huethat falls within a predetermined hue range of one another.

The selectable marker(s) may be displayed partially transparent. Theselectable marker(s) may be periodically displayed and not displayed.

The apparatus may be configured to, based on user input, select a savecommand, save the image to memory with selected features hidden andshown as they were selected on actuation of said save command.

According to a further aspect of the invention, we provide a methodcomprising;

-   -   receiving user input selecting a selectable marker, the        selectable marker marking the position of at least one        identified feature in an image which is hideable from view using        a region of the image surrounding the identified feature,    -   hiding or showing the identified feature in response to said        user input.

The selectable marker may comprise a ring or circular ring thatsurrounds the at least one identified feature. When a plurality offeatures are identified, each feature may be marked by a respectiveselectable marker comprising a ring or circular ring that surrounds therespective feature. The or each selectable markers may include a plussymbol when the identified feature is hidden in the image and includes aminus symbol when the identified feature is shown in the image. The plussymbol and minus symbol may be actuatable. The method may include thestep of showing the hidden feature on actuation of the plus symbol for amarker and hiding the identified feature on actuation of the minussymbol for a marker.

The method may include the step of providing, initially, for display ofthe image without any of the identified features hidden and theselectable markers are initially configured to allow for the hiding of arespective feature in response to said user input selection.

The method may include the step of, when two or more features areidentified, associating the two or more identified features such thatthey are treated as a single feature by determining if said two or moreidentified features satisfy a predefined clustering criterion.

The method may include the step of using a single marker to mark theposition of associated features, wherein the association allows the twoor more associated features to be shown or hidden together in responseto said user input selection.

The mark applied to mark the position of associated features maycomprise a ring that surrounds the associated features.

The predefined clustering criterion may be satisfied if the region ofthe image surrounding the one of the identified features overlaps withthe region of the image surrounding the other of the identifiedfeatures. The predefined clustering criterion may be satisfied if thetwo or more identified features are within a predetermined distance ofone another. The predefined clustering criterion may be satisfied if theregions of the image surrounding the two or more identified features areof a hue that falls within a predetermined hue range of one another.

The selectable marker(s) may be displayed partially transparent. Theselectable marker(s) may be periodically displayed and not displayed.

The method may include the step of, based on user input, saving theimage to memory with selected features hidden and shown as they wereselected on actuation of said save command.

According to a further aspect of the invention, we provide a computerreadable medium comprising computer program code stored thereon, thecomputer readable medium and computer program code being configured to,when run on at least one processor, perform the method of the aboveaspect.

According to an aspect of the invention, we provide means for, inresponse to user input selecting a selectable marker, the selectablemarker marking the position of at least one identified feature in animage which is hideable from view using a region of the imagesurrounding the identified feature, hiding or showing the identifiedfeature.

The selectable marker may comprise a ring or circular ring thatsurrounds the at least one identified feature. Means may be providedfor, when a plurality of features are identified, marking each featureby a respective selectable marker comprising a ring or circular ringthat surrounds the respective feature. The or each selectable markersmay include a plus symbol when the identified feature is hidden in theimage and includes a minus symbol when the identified feature is shownin the image. The plus symbol and minus symbol may be actuatable. Theplus symbol for a marker may allow for the hidden feature to be shownand the minus symbol for a feature may allow for the feature to behidden.

The means may be configured to initially provide for display of theimage without any of the identified features hidden and the selectablemarkers are initially configured to allow for the hiding of a respectivefeature in response to said user input selection.

The means may be configured, when two or more features are identified,associate the two or more identified features such that they are treatedas a single feature by determining if said two or more identifiedfeatures satisfy a predefined clustering criterion.

The means may be configured such that said association allows the two ormore associated features to be shown or hidden together in response tosaid user input selection using a single marker marking the position ofthe associated features.

The mark applied to mark the position of associated features maycomprise a ring that surrounds the associated features.

The predefined clustering criterion may be satisfied if the region ofthe image surrounding the one of the identified features overlaps withthe region of the image surrounding the other of the identifiedfeatures. The predefined clustering criterion may be satisfied if thetwo or more identified features are within a predetermined distance ofone another. The predefined clustering criterion may be satisfied if theregions of the image surrounding the two or more identified features areof a hue that falls within a predetermined hue range of one another.

The selectable marker(s) may be displayed partially transparent. Theselectable marker(s) may be periodically displayed and not displayed.

Means may be provided for, based on user input, saving the image tomemory with selected features hidden and shown as they were selected onactuation of said save command.

We disclose an apparatus comprising: at least one processor; and atleast one memory including computer program code, the at least onememory and the computer program code configured to, with the at leastone processor, cause the apparatus to perform at least the following:

-   -   based on received progressive user swiping input, progressively        select for application to a displayed image during the received        progressive user swiping input, each of a plurality of        differential image effect types.

The received user swiping input may comprise user swiping input along afirst axis and wherein the apparatus is further configured to providefor the application of a different function based on received userswiping input along a second axis, different to the first axis.

The different function may comprise the selection of an image fordisplay from a plurality of time adjacent images.

The apparatus may be further configured, on termination of the userswiping input, to continue to provide for display the image with theimage effect type applied thereto that was last selected prior to thetermination of the user swiping input.

The apparatus may be configured to apply the selected image effect typeto the image.

The apparatus may be configured to apply the selected image effect typeto a lower resolution preview version of a recorded image.

The apparatus may be configured to provide for display a user selectableoption that, when selected, applies the selected image effect type to ahigher resolution version of the recorded image.

The apparatus may be further configured, on termination of the userswiping input, to apply the image effect type that was last selectedprior to the termination of the user swiping input to a higherresolution version of the recorded image.

The plurality of differential image effect types for the image maycomprise at least three differential image effect types.

The apparatus may be configured to receive the particular image from acamera module and is configured to perform the steps of the precedingclaims on receipt of an image or a plurality of time adjacent imagesfrom the camera module.

Selectable image effect types may comprise one or more of the followingimage effect types: face identification, feature removal, red-eyeremoval, image pixelization, making the image monochrome, imageposterization, fisheye deformation of the image, making the imagegreyscale, image edge detection and/or colour shift, or any other imageeffect type.

The different function may comprise adjustment of a parameter associatedwith the selected image effect type.

The apparatus may be configured to provide for display of the particularimage without any of the image effect types applied thereto prior to theinput of user swiping input.

The apparatus may be configured, by way of said user swiping input, toselect for display the particular image without any of the selectableimage effect types applied thereto.

We disclose a method, the method comprising; receiving progressive userswiping input; and progressively selecting for application to adisplayed image during the received progressive user swiping input, eachof a plurality of differential image effect types.

We disclose a computer readable medium comprising computer program codestored thereon, the computer readable medium and computer program codebeing configured to, when run on at least one processor, perform theabove method.

We disclose an apparatus comprising: at least one processor; and atleast one memory including computer program code, the at least onememory and the computer program code configured to, with the at leastone processor, cause the apparatus to perform at least the following:

-   -   receive a first object mask representing a moving part of one of        a plurality of images comprising a sequence of time adjacent        images;    -   receive one or more further object masks, formed from respective        images of the sequence, representing the moving part in        respective different times; and    -   use the object masks in a predetermined clutter criterion to        determine the displaying of the moving part in a composite image        formed from the plurality of time adjacent images.

The predetermined clutter criterion may comprise;

-   -   determining an overdraw parameter based on an area over which        two or more of the object masks overlap;    -   determining, based in the overdraw parameter, whether to display        the moving part as represented in one of the plurality images in        the composite image formed from the plurality of time adjacent        images.

The overdraw parameter may be determined using the area over which twoor more of the object masks overlap in relation to the area over whichthe object masks extend.

The predetermined clutter criterion may further comprise;

-   -   determining a coverage parameter based on the area over which        the object masks extend;    -   determine, based in the coverage parameter, whether to display        the moving part as represented in one of the plurality images in        the composite image formed from the plurality of time adjacent        images.

The coverage parameter may be determined using the area over which theobject masks extend in relation to the total area of one of the images.

The apparatus may be configured to apply the clutter criterionincrementally such that the clutter criterion is reapplied each time itis determined that the apparatus should not display the moving part asrepresented in one of the plurality images in the composite image.

The apparatus may be configured to display the composite image formedafter application of the clutter criterion.

The step of determining the displaying of the moving part in a compositeimage formed from the plurality of time adjacent image may compriseselecting a subset of the plurality of the images in the sequence.

The step of determining the displaying of the moving part in a compositeimage formed from the plurality of time adjacent image may compriseselecting a subset of the object masks for use in extracting the movingpart from the plurality of images in the sequence to assemble acomposite image.

We disclose a method, the method comprising;

-   -   receiving a first object mask representing a moving part of one        of a plurality of images comprising a sequence of time adjacent        images;    -   receiving one or more further object masks, formed from        respective images of the sequence, representing the moving part        in respective different times; and    -   using the object masks in a predetermined clutter criterion to        determine the displaying of the moving part in a composite image        formed from the plurality of time adjacent images.

We disclose a computer readable medium comprising computer program codestored thereon, the computer readable medium and computer program codebeing configured to, when run on at least one processor, perform theabove method.

We disclose an apparatus comprising: at least one processor; and atleast one memory including computer program code, the at least onememory and the computer program code configured to, with the at leastone processor, cause the apparatus to perform at least the following:

-   -   based on a received user selection of one or more of a plurality        of displayed user selectable markers, the user selectable        markers each corresponding to one or more of the plurality of        time adjacent images in a sequence of images, causing an editing        function associated with the one or more images that correspond        to the selected marker.

The causing an editing function may comprise adding or removing the oneor more images that correspond to the selected marker from a set ofselected images which form a composite image from the set of selectedimages.

Each user selectable marker may correspond to one of the time adjacentimages in the sequence of images.

The user selectable markers may be displayed in an order thatcorresponds to the temporal order of the images in the sequence.

The apparatus may be further configured to provide for display thecomposite image formed from the set of selected images in response tothe addition or removal of the one or more images.

The apparatus may be further configured to, based on user input, selectat least one of the user selectable markers as a primary marker, whereinthe image corresponding to the selected primary marker is identified asthe primary image and the other images in the set of selected images areidentified as secondary images, and wherein the apparatus is configuredto provide for display of the primary image in the composite image witha different image parameter to the secondary images.

The image parameter may comprise transparency and the primary image isdisplayed with zero transparency and the secondary images are displayedwith non-zero transparency.

Each one of the secondary images may be displayed in the composite imagewith a non-zero transparency that is determined based on its position inthe set of selected images relative to the primary image.

A first secondary image that is temporally closer to the primary imagein the set of selected images than a second secondary image, may beprovided for display with a lower transparency than the second secondaryimage.

The apparatus may be configured to include, initially, all of thereceived time adjacent images in the set of selected images.

The apparatus may include a camera for capturing the plurality of timeadjacent images that comprise a sequence of images.

We disclose a method, the method comprising

-   -   receiving a user selection of one or more of a plurality of        displayed user selectable markers, the user selectable markers        each corresponding to one or more of the plurality of time        adjacent images in a sequence of images;    -   causing an editing function associated with the one or more        images that correspond to the selected marker, in response to        receiving said user selection.

We disclose a computer readable medium comprising computer program codestored thereon, the computer readable medium and computer program codebeing configured to, when run on at least one processor, perform theabove method.

We disclose an apparatus comprising: at least one processor; and atleast one memory including computer program code, the at least onememory and the computer program code configured to, with the at leastone processor, cause the apparatus to perform at least the following:

-   -   based on received progressive user swiping input, progressively        adjust a first image parameter in response to the amount of        progressive user swiping input relative to a first axis and        simultaneously progressively adjust a second image parameter,        different to the first image parameter, in response to the        amount of progressive user swiping input relative to a second        axis; and    -   provide for display of said image with modifications made        thereto based on the first image parameter and second image        parameter.

The apparatus may be configured to determine if the amount of userswiping relative to the first axis or second axis is greater than arespective threshold amount and, if it is not greater than therespective threshold amount, no adjustment of the first image parameteror respective second image parameter is effected.

First and second image parameters may be selected from: imagebrightness, image contrast, image white balance, image saturation, imagehue, image highlight level, image low light level, image mid-tone levelor any other image parameter.

The apparatus may be configured to provide for progressive display,during said user swiping input, of said image with modifications madethereto using the first and second image parameters.

Said image may comprise a composite image formed from a plurality oftime adjacent images and said first image parameter comprises theselection of one of the plurality of time adjacent images as a primaryimage and said second image parameter comprises a transparency level forapplication to the time adjacent images.

The second image parameter may not be applied to the selected primaryimage.

We disclose a method, the method comprising

-   -   receiving progressive user swiping input,    -   progressively adjusting a first image parameter in response to        the amount of progressive user swiping input relative to a first        axis and simultaneously progressively adjusting a second image        parameter, different to the first image parameter, in response        to the amount of progressive user swiping input relative to a        second axis; and    -   providing for display of said image with modifications made        thereto based on the first image parameter and second image        parameter.

We disclose a computer readable medium comprising computer program codestored thereon, the computer readable medium and computer program codebeing configured to, when run on at least one processor, perform theabove method.

We disclose an apparatus comprising:

-   -   at least one processor; and at least one memory including        computer program code, the at least one memory and the computer        program code configured to, with the at least one processor,        cause the apparatus to perform at least the following:    -   progressively adjust a first image parameter of a live image in        response to progressive user swiping input; and    -   provide for display of said live image with progressive        modifications made thereto based on progressive adjustment of        the first image parameter.

The apparatus may be further configured to

-   -   allow user swiping input along a second axis, different to the        first axis;    -   based on user swiping input along a first axis, progressively        adjust a first image parameter in response to the amount of user        swiping input relative to the first axis and, based on user        swiping input along a second axis, different to the first axis,        simultaneously    -   progressively adjust a second image parameter, different to the        first image parameter, in response to the amount of user swiping        input relative to the second axis; and progressively display        said live image with modifications made thereto based on the        first image parameter and second image parameter.

The apparatus may include a camera for capturing said live image.

We disclose a method, the method comprising

-   -   receiving progressive user swiping input;    -   progressively adjusting a first image parameter of a live image        in response to said progressive user swiping input; and    -   providing for display of said live image with progressive        modifications made thereto based on progressive adjustment of        the first image parameter.

We disclose a computer readable medium comprising computer program codestored thereon, the computer readable medium and computer program codebeing configured to, when run on at least one processor, perform theabove method.

The apparatus in any of the disclosed apparatuses may comprise aportable electronic device, a laptop computer, a mobile phone, aSmartphone, a tablet computer, a personal digital assistant, a digitalcamera, a watch, a server, or a module/circuitry for one or more of thesame.

The apparatus of any of the disclosed apparatuses may comprise anon-portable electronic device, a desktop computer, a monitor, a server,or a module/circuitry for one or more of the same.

The present disclosure includes one or more corresponding aspects,examples or features in isolation or in various combinations whether ornot specifically stated (including claimed) in that combination or inisolation. Corresponding means and corresponding functional units (e.g.application loader, journal creation enabler . . . ) for performing oneor more of the discussed functions are also within the presentdisclosure.

Corresponding computer programs for implementing one or more of themethods disclosed are also within the present disclosure and encompassedby one or more of the described examples.

The above summary is intended to be merely exemplary and non-limiting.

BRIEF DESCRIPTION OF THE FIGURES

A description is now given, by way of example only, with reference tothe accompanying drawings, in which:

FIG. 1 illustrates an example apparatus according to the presentdisclosure;

FIG. 2 illustrates another example apparatus

FIG. 3 illustrates another example apparatus

FIG. 4 illustrates an example apparatus having a keyboard

FIG. 5 illustrates an example touch screen device

FIGS. 6 a-6 d illustrate a first example of a first aspect of theinvention

FIGS. 7 a-7 c illustrates a second example of a first aspect of theinvention

FIGS. 8 a-8 d illustrates a third example of a first aspect of theinvention

FIG. 9 illustrates a flowchart according to a method of the presentdisclosure;

FIGS. 10 a-10 b shows an example of a cluttered and uncluttered image

FIGS. 11-13 illustrate an example of a second aspect of the invention

FIG. 14 illustrates a flowchart according to a method of the presentdisclosure;

FIGS. 15 a and 15 b illustrate an example of the invention

FIG. 16 illustrates a flowchart according to a method of the presentdisclosure;

FIGS. 17 a-d illustrate a first embodiment of the invention

FIGS. 18 a-b illustrate a second embodiment of the invention

FIGS. 19 a-b illustrate a third embodiment of the invention

FIGS. 20 a-b illustrate a fourth embodiment of the invention

FIG. 21 illustrates a flowchart according to a method of the presentdisclosure;

FIGS. 22 a-b illustrate an example of a fifth aspect of the invention

FIG. 23 illustrates a flowchart according to a method of the presentdisclosure;

FIGS. 24 a-c illustrate an example according to a sixth aspect of theinvention;

FIG. 25 illustrates a flowchart according to a method of the presentdisclosure; and

FIG. 26 illustrates schematically a computer readable medium providing aprogram.

DESCRIPTION OF EXAMPLE ASPECTS

The application of image effects may involve the user making a selectionto choose the type of image effect to apply to an image from a menu,waiting for the image to be rendered with the selected effect, and thenbeing presented with options for adjusting a parameter associated withthe effect type. For example, the effect type may comprise pixelizationof the image and the parameter may be the size of the pixels used in thepixelization. Alternatively, the effect type may be transparency and theparameter may be the level of transparency.

If the user does not like the look of the image once the effect has beenapplied they typically have to select an “undo” option to return to aprevious version of the image. It would be useful for the user to seethe result of the application of different effect types to an imagequickly and easily. They would then be able to make a quick assessmentof which image effect type they wish to apply to the image.

FIG. 1 shows an apparatus 100 comprising a processor 110, memory 120,input I and output O. In this example only one processor and one memoryare shown but it will be appreciated that other examples may utilisemore than one processor and/or more than one memory (e.g. same ordifferent processor/memory types). The apparatus 100 may be anapplication specific integrated circuit (ASIC) for a portable electronicdevice. The apparatus 100 may also be a module for a device, or may bethe device itself, wherein the processor 110 is a general purpose CPUand the memory 120 is general purpose memory.

The input I allows for receipt of signalling to the apparatus 100 fromfurther components. The output O allows for onward provision ofsignalling from the apparatus 100 to further components. In this examplethe input I and output O are part of a connection bus that allows forconnection of the apparatus 100 to further components. The processor 110is a general purpose processor dedicated to executing/processinginformation received via the input I in accordance with instructionsstored in the form of computer program code on the memory 120. Theoutput signalling generated by such operations from the processor 110 isprovided onwards to further components via the output O.

The memory 120 (not necessarily a single memory unit) is a computerreadable medium (such as solid state memory, a hard drive, ROM, RAM,Flash or other memory) that stores computer program code. This computerprogram code stores instructions that are executable by the processor110, when the program code is run on the processor 110. The internalconnections between the memory 120 and the processor 110 can beunderstood to provide active coupling between the processor 110 and thememory 120 to allow the processor 110 to access the computer programcode stored on the memory 120.

In this example the input I, output O, processor 110 and memory 120 areelectrically connected internally to allow for communication between therespective components I, O, 110, 120, which in this example are locatedproximate to one another as an ASIC. In this way the components I, O,110, 120 may be integrated in a single chip/circuit for installation inan electronic device. In other examples one or more or all of thecomponents may be located separately (for example, throughout a portableelectronic device such as devices 200, 300, or through a “cloud”, and/ormay provide/support other functionality.

One or more examples of the apparatus 100 can be used as a component foranother apparatus as in FIG. 2, which shows a variation of apparatus 100incorporating the functionality of apparatus 100 over separatecomponents. In other examples the device 200 may comprise apparatus 100as a module (shown by the optional dashed line box) for a mobile phoneor PDA or audio/video player or the like. Such a module, apparatus ordevice may just comprise a suitably configured memory and processor.

The example apparatus/device 200 comprises a display 240 such as aLiquid Crystal Display (LCD), e-Ink, or touch-screen user interface. Thedevice 200 is configured such that it may receive, include, and/orotherwise access data. For example, device 200 comprises acommunications unit 250 (such as a receiver, transmitter, and/ortransceiver), in communication with an antenna 260 for connection to awireless network and/or a port (not shown). Device 200 comprises amemory 220 for storing data, which may be received via antenna 260 oruser interface 230. The processor 210 may receive data from the userinterface 230, from the memory 220, or from the communication unit 250.Data may be output to a user of device 200 via the display device 240,and/or any other output devices provided with apparatus. The processor210 may also store the data for later user in the memory 220. The devicecontains components connected via communications bus 280.

The communications unit 250 can be, for example, a receiver,transmitter, and/or transceiver, that is in communication with anantenna 260 for connecting to a wireless network and/or a port (notshown) for accepting a physical connection to a network, such that datamay be received via one or more types of network. The communications (ordata) bus 280 may provide active coupling between the processor 210 andthe memory (or storage medium) 220 to allow the processor 210 to accessthe computer program code stored on the memory 220.

The memory 220 comprises computer program code in the same way as thememory 120 of apparatus 100, but may also comprise other data. Theprocessor 210 may receive data from the user interface 230, from thememory 220, or from the communication unit 250. Regardless of the originof the data, these data may be outputted to a user of device 200 via thedisplay device 240, and/or any other output devices provided withapparatus. The processor 210 may also store the data for later user inthe memory 220.

Device/apparatus 300 may be an electronic device (including a tabletpersonal computer), a portable electronic device, a portabletelecommunications device, or a module for such a device. The apparatus100 can be provided as a module for device 300, or even as aprocessor/memory for the device 300 or a processor/memory for a modulefor such a device 300. The device 300 comprises a processor 385 and astorage medium 390, which are electrically connected by a data bus 380.This data bus 380 can provide an active coupling between the processor385 and the storage medium 390 to allow the processor 385 to access thecomputer program code.

The apparatus 100 in FIG. 3 is electrically connected to an input/outputinterface 370 that receives the output from the apparatus 100 andtransmits this to the device 300 via data bus 380. Interface 370 can beconnected via the data bus 380 to a display 375 (touch-sensitive orotherwise) that provides information from the apparatus 100 to a user.Display 375 can be part of the device 300 or can be separate. The device300 also comprises a processor 385 that is configured for generalcontrol of the apparatus 100 as well as the device 300 by providingsignalling to, and receiving signalling from, other device components tomanage their operation.

The storage medium 390 is configured to store computer code configuredto perform, control or enable the operation of the apparatus 100. Thestorage medium 390 may be configured to store settings for the otherdevice components. The processor 385 may access the storage medium 390to retrieve the component settings in order to manage the operation ofthe other device components. The storage medium 390 may be a temporarystorage medium such as a volatile random access memory. The storagemedium 390 may also be a permanent storage medium such as a hard diskdrive, a flash memory, or a non-volatile random access memory. Thestorage medium 390 could be composed of different combinations of thesame or different memory types.

FIGS. 4 and 5 illustrate examples of the apparatus which is a portableelectronic device 400, 500 such as a smartphone or PDA, in use. Thedevice 400 in this example includes a keyboard for receiving user input.The device 500 includes a touch screen display for receiving user input.In particular, the device 500 provides user selectable icons and othermanipulatable graphics that allow a user to touch an area of the screenassociated with the icon or graphic to actuate it. The device 500 maychange the appearance or position of the icons or graphics in responseto the user input, as will be known to those in the art. Further, thedevice 500 is receptive to gestures, such as swipes across the screen,the tracing of paths along the surface, contact with the screen andremoval of contact with the screen, which can be configured to providedifferent user input.

The exemplary embodiments described herein will be referred to as beingperformed by a processor of a mobile device having a touch screen input,but it will be appreciated that the embodiments could be performed usingany of the apparatus, processors, modules or devices shown in FIGS. 1 to5. Also, in other embodiments, one or more of the described aspects maybe performed using input via a peripheral device, eg a mouse, and notnecessarily via a touch screen. In the case of input via a peripheraldevice such as a mouse it will be appreciated that the progressive userswiping comprises translational/lateral (linear, curved or otherwise)movement of the mouse during a selection actuation.

FIGS. 6 a to 6 d shows a mobile device 600 having a processor and memoryand a touch screen display 605 for receiving user input. The processoris configured to provide a plurality of image effect types for applyingto an image. In this embodiment three image effect types are availablefor selection. It will be appreciated that 4, 5, 6, 7, 8, 9, 10, or moreimage effect types may be provided for selection. The three differentialimage effect types available comprise a monochrome effect type, apixelated effect type and a negative effect type.

In this example the mobile device 600 includes a camera (not visible)that has been used to take a photograph of a tree. The image from thecamera is recorded in memory at a native resolution of the camera. InFIG. 6 a the user has entered an image effect type preview mode. In thismode, the processor is configured to provide an image region 610 and auser input region 615 for a user to use in providing user input to thedevice 600. In the image effect type preview mode, a lower resolutionversion of the recorded image is displayed in the image region 610. Thefirst of the three image effect types is selected by default and appliedby the processor to the lower resolution version of the image displayedin image region 610. In this embodiment, as shown in FIG. 6 a, the firstimage effect type is a monochrome effect type. Accordingly, the lowerresolution version of the recorded image is converted from its originalcolour form to a monochrome form and displayed by the processor.

In FIG. 6 a, the user 620 places their finger on the user input region615. FIGS. 6 a to 6 c show the result of the user progressively swipingtheir finger across the user input region 615 from a first point 625 toa second point 630. It will be appreciated that progressive user swipingcomprises translational/lateral (linear, curved or otherwise) movementof user's finger (or stylus/peripheral device or touch screen inputtool) while in contact with the touch screen input region 715. In thecase of input via a peripheral device such as a mouse it will beappreciated that the progressive user swiping comprisestranslational/lateral (linear, curved or otherwise) movement of themouse during a selection actuation.

FIG. 6 b shows the user's progressive swiping input reaching a point onthe user input region 615 between points 625 and 630. At this point theprocessor selects a second of the three image effect types. In thisinstance, the second image effect type comprises the pixelated effecttype. Accordingly, the processor stops applying the initially appliedmonochrome effect type and applies the pixelated effect type to thelower resolution version of the image. Accordingly, the lower resolutionversion of the recorded image is converted to a pixelated form anddisplayed by the processor in the image region 610 during the user'sprogressive swiping input.

FIG. 6 c shows the user's progressive swiping input reaching the secondpoint 630. At this point the processor selects a third of the threeimage effect types. In this instance, the third image effect typecomprises the negative effect type. Accordingly, the processor stopsapplying the pixelated effect type and applies the negative effect typeto the lower resolution version of the image. Accordingly, the lowerresolution version of the recorded image is converted to a negative formand displayed by the processor in the image region 710 during the user'sprogressive swiping input.

At point 630 the user terminates the progressive swiping input byremoving their finger as shown in FIG. 6 d from the user input region615. The processor continues to display the last selected (in thisinstance third image effect type) applied to the lower resolutionversion of the recorded image. The user can then select whether theywish to apply the selected image effect type at a higher resolution.This may be achieved by the user tapping the image region 610 of thedisplay. However, in this embodiment the processor displays a selectableoption 635 which may have an informative icon or description to identifyits function to a user, such as “Save?”. On actuation of the selectableoption by the user touching the screen 605 over the displayed selectableoption 635, the processor applies the selected image effect type to therecorded image and displays the resulting image to the user. Theprocessor may also save a modified version of the recorded image in thememory or overwrite the recorded image.

Rather than select the selectable option the user may decide to returnto a previously applied effect. Accordingly the user can again contactthe input region 615 and progressive scroll back to point 625 and theprocessor is configured to progressively select the second and then thefirst image effect types and display the lower resolution version of theimage with the selected effect types applied as they are selected. Ontermination of this second progressive swiping input, the processor mayagain display the selectable option 635 so that the user can choose toapply the selected (first, monochrome image effect type) effect to therecorded image.

FIG. 7 a-7 c shows a second embodiment in which the image region 730covers the whole display 705. This embodiment does not utilize a userinput region and instead user input is received by user input applied atany point on the display 705. The processor is configured to display alow resolution version of the recorded image in the image effect previewmode shown in FIGS. 7 a to 7 c. In this embodiment, the progressiveselection of image effect types by progressive user swiping input issimilar to the previous embodiment. However, in this embodiment, theprocessor is configured to cycle through and select the three imageeffect types more than once during a progressive user swiping input frompoint 725 to point 730. Thus, a shorter movement by the user that formspart of the swiping input causes the processor to select the next imageeffect type. For example, in moving from point 725 to point 730, theprocessor may select the first image effect type, followed by thesecond, then the third and then switching back to the first, then to thesecond again and then to the third by the time the progressive userswiping input reaches point 730. It will be appreciated that differentswitching scales, i.e. the distance moved during the user swiping inputthat will cause the processor switch between the image effect types, maybe used for the progressive user swiping input. Again, like in theprevious embodiment, the processor progressively displays the selectedeffect applied to a lower resolution “preview” version of the recordedimage.

In the second embodiment four image effect types are provided, namely,outline effect, red-eye removal, saturation boost and face detectiontagging. FIG. 7 a shows the device 700 in the image effect type previewmode as a user places their finger on the display at the point 725 tobegin a user swiping input. In this embodiment the processor does notinitially apply one of the four image effect types to the lowerresolution version of the image for display. Instead, it displays alower resolution version of the image without any of the image effectstypes applied.

FIG. 7 b shows the user having moved their finger to a point that causesthe processor to select the first image effect type, namely the outlineeffect type. Accordingly, a lower resolution preview is displayed whenthe first effect is selected during the user input swiping. The user hasdecided that they like the outline effect and have accordinglyterminated the swiping input by lifting their finger from the display asshown by arrow 740. In this embodiment, the processor is configured todetect the termination of the swiping input and then automaticallydisplay the selected image effect type (in this example the first,outline effect) applied to a higher resolution version of the image.FIG. 7 c shows the display of a higher resolution version of the imagewith the outline effect applied. The user did not choose to continue theprogressive swiping input to progressively view the red-eye removal,saturation boost and face detection tagging effect types.

In a third embodiment, shown in FIGS. 8 a to 8 c, the image is a burstimage, which comprises a plurality of time adjacent images taken insuccession to form a sequence of images. Burst images are typically usedto capture movement. This embodiment operates in a similar manner to thesecond embodiment. However, the processor is configured to differentiatebetween user swiping input in a first direction and user swiping inputin a second direction. In this instance, the first direction is ahorizontal direction and the second direction is a vertical directionwith respect to the display 805. FIGS. 8 a and 8 b show a user swipinginput in the first direction. The processor is configured toprogressively select the images that make up the sequence of images fordisplay. Thus, FIG. 8 a shows the first image in the sequence of imagesin which a diver is standing on a diving board. Progressive swiping userinput in the horizontal direction to the point in FIG. 8 b has causedthe processor to progressively select and display the second image inthe sequence, followed by the third image in the sequence and now thefourth image in the sequence, as shown in FIG. 8 b. In the fourth image,shown in FIG. 8 b, the diver has jumped up from the diving board and isstarting their descent to the pool.

The user decides that they like this fourth image in the sequence andwould like to preview an image effect type applied thereto. Thus, asshown in FIG. 8 c, the user continues to provide user swiping input butnow in a vertical direction. The processor is configured to cyclethrough the image effect types, as discussed in previous embodiments,and progressively apply those effects to the selected fourth image inthe sequence for display. In FIG. 8 c the user has paused the userswiping input with a sixth image effect type out of a possible ten imageeffect types selected. Accordingly the lower resolution preview isdisplayed following the display of the previously selected five imageeffect types. The sixth image effect type comprises an edge enhanceimage effect. The user may decide that they like this image effect typebut would prefer that it was applied to an image four images furtheralong in the sequence of images. Accordingly, the processor isconfigured, on further user swiping input in the first direction, toprogressively select and display the fifth image in the sequence withthe edge enhance image effect applied thereto, then the sixth image inthe sequence with the edge enhance image effect applied thereto and soon until the progressive user swiping input causes the processor toselect and display the ninth image with the selected sixth image effecttype applied, as shown in FIG. 8 d.

FIG. 9 shows a flow diagram illustrating the steps of receiving userswiping input 900, progressively selecting an image effect type from aplurality of differential image effect types 910, progressivelyproviding for display of the selected image effect type applied to animage during said progressive user swiping input 920 and isself-explanatory.

FIGS. 10 to 14 shows an embodiment of a second aspect of the invention.As discussed above a burst image comprises a plurality of time adjacentimages that form a sequence of images. Typically, a camera will have amode to take a plurality of images in succession which can show how amoving object moves through the captured scene over the plurality ofimages. A composite image can be formed by combining the plurality ofimages in the sequence into a single image, perhaps by superimposing,blending or overlaying each image on top of one another.

Composite images can appear to be cluttered, especially when there islittle difference between the images in the sequence. This may be due tothe moving object in the scene moving relatively slowly or could bebecause the camera captures the sequence of images very quickly.

It would be advantageous if composite images could be formed that arenot cluttered and therefore depict the movement of a moving object inthe scene more clearly. The following embodiment comprises an example ofhow the sequence of images can be processed to form an unclutteredcomposite image.

FIG. 10 a shows a composite image that is cluttered due to theoverlapping of its component images from the sequence of images. FIG. 10b shows a composite image formed from a subset of the sequence of imagesthat results in an uncluttered composite image.

In a first embodiment, an apparatus, which may be a mobile device or anyother apparatus as discussed above, includes a camera that is adapted tocapture a plurality of time adjacent images that form a sequence ofimages. A processor of the apparatus is configured to analyse theplurality of images to determine how a composite image is formed fromthe sequence of images.

In summary, the processor is configured to receive a plurality of imagesthat form a sequence of images, assess the plurality of images using aclutter criterion, if the clutter criterion is not met, form a subset ofthe plurality of images and then provide for display of a compositeimage formed from the subset of images.

In particular, a moving part of the sequence of images is identifiedusing a moving parts detection and background extraction algorithm, suchas using the median image to detect the background, and then use thecolour and intensity distance from the background image to identifymoving parts. The skilled man will be familiar with such an algorithmand therefore it will not be discussed here in more detail. A movingpart of the sequence of images comprises a feature common to two or moreof the plurality of images that changes its shape and/or position in thescene over the two or more images.

The algorithm returns an object mask comprising an area in the imageswhere the moving object is present. The algorithm produces an objectmask for each of the plurality of images in which the moving object ispresent.

In FIG. 11 a an example image is shown that is considered to becluttered. The composite image is formed from five component images thatform a sequence of images. The sequence of images have captured themovement of a ball following a curved path from the bottom left cornerof the first image of the sequence of images to the top middle region bythe third image of the sequence of images and then down to the bottomright corner of the image by the fifth and final image in the sequenceof images.

The algorithm is able to identify the ball as comprising a moving objectover the plurality of images. The algorithm returns a first object maskfor the first image in the sequence that comprises an area that definesthe position of the ball in the first image. A depiction of the firstobject mask 1205 is shown in FIG. 12 a. Likewise, the algorithm returnsa second object mask for the second image in the sequence that comprisesan area that defines the position of the identified moving object (theball) in the second image. A depiction of the second object mask 1210 isshown in FIG. 12 b. The algorithm returns a third object mask for movingobject identified in the third image and so on for each of the images inthe sequence in which the moving object can be identified.

The processor is configured to use the object masks with a cluttercriterion that is used to determine how the plurality of images are usedin the construction of a composite image.

In this embodiment, the clutter criterion considers a measure of theoverlap between the object masks and a measure of the coverage areacovered by the object masks. In a second embodiment, the cluttercriterion only considers the measure of overlap. In a third embodiment,the clutter criterion only considers the measure of coverage area.

The measure of overlap is determined by calculation of an overdrawparameter. The overdraw parameter comprises a ratio of the total areacovered by overlapping portions of all of the object masks to the totalarea covered by the object masks. The area 1350 is a region where thefirst object mask 1305 overlaps with the second object mask 1310. Thearea 1355 is a region where the second object mask 1310 overlaps withthe third object mask 1315. The area 1360 is a region where the thirdobject mask 1315 overlaps with the fourth object mask 1320. The area1365 is a region where the fourth object mask 1320 overlaps with thefifth object mask 1325. The sum of the areas 1355, 1360, 1365, 1370comprises the total overlapping area. The total area over which theobject masks extend (without counting any area within overlapping areasmore than once) comprises the total mask area. Thus, the total mask areacomprises the area represented by the dashed line 1370. The overdrawparameter is calculated using a ratio of the total overlapping area tothe total mask area.

To determine whether this part of the clutter criterion is satisfied theoverdraw parameter is compared to a predefined overdraw threshold. Inthis embodiment, the overdraw threshold is set at 0.5, although it willbe appreciated that other values could be used or the overdraw thresholdcould be user adjustable. If the overdraw parameter is greater than theoverdraw threshold then this part of the clutter criterion is notsatisfied. If the overdraw parameter is less than or equal to theoverdraw threshold then this part of the clutter criterion is satisfied.

The measure of the area covered by the object masks is determined bycalculation of a coverage parameter. The coverage parameter comprises aratio of the total mask area to the total area of one of the images inthe sequence of images.

To determine whether this part of the clutter criterion is satisfied thecoverage parameter is compared to a predefined coverage threshold. Inthis embodiment, the coverage threshold is set at 0.5, although it willbe appreciated that other values could be used or the coverage thresholdcould be user adjustable. If the coverage parameter is greater than thecoverage threshold then this part of the clutter criterion is notsatisfied. If the coverage parameter is less than or equal to thecoverage threshold then this part of the clutter criterion is satisfied.

If both the measure of overlap and measure of coverage is less than orequal to their respective thresholds then the clutter criterion issatisfied and all of the plurality images in the sequence are used toform the composite image. The processor displays the composite image.

If either or both of the measure of overlap and measure of coverage isgreater than their respective thresholds the clutter criterion is notsatisfied. In this instance, a subset of the plurality of images in thesequence is formed. In this embodiment the subset is formed by theremoval of one of the images in the sequence. The image that is removedis chosen at random (although other methods could be employed to selectwhich image or images are not included in the subset). FIG. 11 b showsthe second image in the sequence of images has not been included in afirst subset.

The clutter criterion is then reassessed using only the object masksassociated with the subset of images. Accordingly, the total overlappingarea now comprises the total of areas 1360 and 1365. The total mask areacomprises the area 1375 plus the area 1380 shown by the dashed lines

In this example, the clutter criterion is still not satisfied.Accordingly, a further subset is formed that includes fewer of theplurality of images than the previous, first subset. FIG. 11 c showsthat the fourth image in the sequence has not been included in thefurther subset. The further subset therefore comprises the first, thirdand fifth images of the sequence of images.

The clutter criterion is then reassessed using only the object masksassociated with the further subset of images. Accordingly, the totaloverlapping area is now zero. The total mask area comprises the area ofthe first object mask 1305 plus the area of the third object mask 1315plus the area of the fifth object mask 1325.

The clutter criterion is now satisfied as the overdraw parameter is notgreater than the overdraw threshold and the coverage parameter is notgreater than the coverage threshold. The processor then provides fordisplay of a composite image formed using the images of the furthersubset. FIG. 11 c shows the resulting composite image which is formedfrom the first, third and fifth images in the sequence of images.

In a further embodiment, the apparatus may start by forming a subset oftwo of the images in the sequence of images. The processor then receivesthe object masks from the algorithm. The clutter criterion is thenapplied and, if it is satisfied, an image from the sequence of images isadded to the subset. The clutter criterion is then reapplied to thelarger subset. This method iteratively builds up a set of images from asequence of images until the clutter criterion is no longer satisfied.It then provides for display of a composite image using the last subsetof images that satisfied the clutter criterion. This is in contrast tothe previous method, which started with all of the images in thesequence and iteratively removed images from a working set until theclutter criterion was met.

FIG. 14 shows a flow diagram illustrating the steps of receiving a firstobject mask representing a moving part of one of a plurality of imagescomprising a sequence of time adjacent images 1400, receiving one ormore further object masks, formed from respective images of thesequence, representing the moving part in respective different times1405, and using the object masks in a predetermined clutter criterion todetermine the displaying of the moving part in a composite image formedfrom the plurality of time adjacent images 1410 and is self-explanatory.

FIGS. 15 a and 15 b show a first embodiment of a third aspect of theinvention. As discussed above, a burst image comprises a plurality oftime adjacent images that form a sequence of images. Typically, a camerawill have a mode to take a plurality of images in succession which canshow how a moving object moves through the captured scene over theplurality of images. Thus, the path or other changes of a moving objectare captured in a series of still images. A composite image can beformed by combining the plurality of images in the sequence into asingle image, perhaps by superimposing, blending or overlaying eachimage on top of one another. The moving object then appears in each ofits shapes/positions in a single image to give the impression ofmovement in a still image.

Composite images can appear to be cluttered, especially when there islittle difference between the images in the sequence. This may be due tothe moving object in the scene moving relatively slowly or could bebecause the camera captures the sequence of images very quickly.Further, the style of a composite image can be changed by the number anddegree of movement of a moving object in the scene between componentimages.

It would be advantageous if composite images could be formed easily fromselected images from a sequence of images. The following embodimentcomprises an example of an apparatus for easy selection of the componentimages that form a composite image.

A device having a processor and a camera may capture a burst mode imagecomprising a plurality of images in sequence. Alternatively, theplurality of images may be stored or taken at an earlier time. Theprocessor is configured to assemble the plurality of images into acomposite image. FIG. 15 a shows such a composite image 1500, formedfrom six images taken in quick succession of a person performing thelong jump at an athletics event. The six component images thus show theperson in various positions during their take-off, flight through theair and landing as well as in different shapes as they move their bodyduring each phase. It will be appreciated that any number of imagescould be present in the sequence of images captured in the burst mode.

In FIG. 15 a, all of the captured images (six) are used to form thecomposite image 1500. The processor displays a plurality of selectablemarkers 1505. The processor is configured to display a selectable marker1505 a, 1505 b, 1505 c, 1505 d, 1505 e and 1505 f for each image in thesequence of images. The processor is configured to associate each of theselectable markers with one of the images that form the composite image.Further, the processor is able to determine the temporal order of thesequence of images by using a time stamp or index number stored inmetadata associated with each of the plurality of images. Theassociation between the selectable markers and the images is thereforeordered such that the order in which the selectable markers arepresented corresponds to the temporal order of the images. Theselectable markers are presented in a longitudinal strip along thebottom of the screen as shown in the figures. The selectable marker 1505a is associated with the first image 1510 a. The selectable marker 1505b is associated with the second image 1510 b. The selectable marker 1505c is associated with the third image 1510 c. The selectable marker 1505d is associated with the fourth image 1510 d. The selectable marker 1505e is associated with the fifth image 1510 e. The selectable marker 1505f is associated with the sixth image 1510 f.

The processor is also adapted to apply transparency effects to theplurality of images that form the composite image 1500. This is usefulwhen only a selection of the component images have the transparencyeffect applied. In this embodiment the transparency effect is applied toall of the images except to one of the images that is designated theprimary or “hero” image. The primary image is selected by the processorto be the final image in the sequence by default. The primary image istherefore emphasised relative to the other images in the sequence whichcan enhance the illusion of motion in the composite image.

FIG. 15 a shows the sixth selectable marker 1505 f having a ring 1515displayed around it. This provides feedback to a user that the sixthimage 1510 f is selected as the primary image. It will also be seen thatthe first to sixth selectable markers 1505 a, 1505 b, 1505 c, 1505 d,1505 e and 1505 f are all displayed as solid circles. This providesfeedback to the user that all of the six composite images of thesequence of images are displayed in the composite image 1500.

The processor is configured to receive user input from the user by wayof an interaction with the selectable markers 1505 a, 1505 b, 1505 c,1505 d, 1505 e and 1505 f to cause an editing function, which in thisembodiment comprises modifying the composite image by adding or removingrespective component images therefrom. Thus, after each selection by auser of a selectable marker, the processor displays the modifiedcomposite image to show the result of adding or removing the selectedcomponent image.

FIG. 15 b shows the user interface after several user selections as wellas a correspondingly modified composite image 1520. In particular, theprocessor has received user selection comprising a selection of thesecond selectable marker 1505 b. In response to this user selection theprocessor has removed the second component image 1510 b from the set ofimages that are used to form the composite image. The processor thendisplayed a modified composite image having the second component image1510 b removed therefrom. The processor has changed the appearance ofthe selectable marker 1505 b to show that the second image 1510 b is nolonger displayed in the composite image. The second selectable marker1505 b is therefore shown as a hollow circle.

The user then made a user selection of the fourth selectable marker 1505d. Likewise, the processor removed the fourth component image 1510 dfrom the set of images that form the composite image and displayed amodified composite image using the modified set of images. Further, thefourth selectable marker 1505 f was modified to be shown as a hollowcircle.

The user then made a user selection of the sixth selectable marker 1505f. Likewise, the processor removed the sixth component image 1510 f fromthe set of images that form the composite image and displayed a modifiedcomposite image using the modified set of images. Further, the sixthselectable marker 1505 f was modified to be shown as a hollow circle.

The user then made a user selection of the first selectable marker 1505a. Likewise, the processor removed the first component image 1510 a fromthe set of images that form the composite image and displayed a modifiedcomposite image using the modified set of images. Further, the firstselectable marker 1505 a was modified to be shown as a hollow circle.The user then did not like the displayed resulting composite image anddecided to reinstate the first image 1510 a in the composite image.Accordingly the processor received a user selection of the firstselectable marker 1505 a. The processor then added the correspondingimage, first image 1510 a, to the set of images that form the compositeimage and displayed a modified composite image using the modified set ofimages. The first selectable marker 1505 a was modified to be shown as asolid circle once again. The result of the above selections is shown inFIG. 15 b, which shows a composite image 1520 that is formed from thefirst, third and fifth component images 1510 a, 1510 c, 1510 e in thesequence of images.

When the user selected the sixth selectable marker 1505 f thecorresponding sixth image 1510 f was removed from the set of images thatform the composite image. The sixth image 1510 f was set to be theprimary image by default in FIG. 15 a. Accordingly, when the userselected the sixth selectable marker 15105 f the designation of aprimary image was also removed by the processor. The processor thereforeno longer applied any transparency effects to the composite image.During the above described user selections, the user also made a gestureto select the fifth image 1510 e as the primary image. In thisembodiment the user applies a double tap on the fifth selectable marker1505 e. The processor is adapted to receive this user selection anddesignate the corresponding fifth image 1510 e as the primary image.Accordingly, in forming the composite image 1520, the processor providesfor the first and third images 1510 a, 1510 c to have the transparencyeffect applied thereto and the fifth image 1510 e to not have thetransparency effect applied.

In a second embodiment, the processor is adapted to associate each ofthe selectable markers with a plurality of images in the sequence ofimages. Thus, where the plurality of images in the sequence totals nine,a first selectable marker may be associated with the first, fourth andseventh images in the sequence. A second selectable marker may beassociated with the second, fifth and eighth images in the sequence anda third selectable marker may be associated with the third, sixth andninth images in the sequence. User selection of one of the markers willtherefore add or remove all of the associated images from the set ofimages that form the composite image.

In a third embodiment, the processor does not initially include any ofthe plurality of images in the sequence in the set of images that areused to form the composite image. Therefore, all of the user selectablemarkers would be displayed as hollow circles (if using the same feedbackas in the previous embodiment, which will be appreciated as notessential). The processor therefore adds images to the set of imagesthat form the composite image with each user selection of a selectablemarker (or removes an image if it has already been added). Thisembodiment can be advantageous as the processor can be configured toinitially display instructions for a user on how to use the selectablemarkers to form a composite image. When the user makes their firstselection of a selectable marker, the instructions are removed and thecomposite image is displayed in their place.

FIG. 16 shows a flow diagram illustrating the steps of receiving aplurality of images that form a sequence 1600, providing for display aplurality of selectable markers 1605, associating the plurality ofselectable markers with the plurality of images 1610, based on userselection of one or more of the user selectable markers, add or removeimages from a set of selected images for forming a composite imageformed from the set of selected images 1615 and is self-explanatory.

Image editing can be used to modify images. One type of image editingcomprises feature removal. Feature removal can be used to hide featureswithin an image so that it appears the feature was not present in theoriginally captured scene. For example, a user may wish to capture animage of a boat passing in front of a sunset. Just as the boat reachesthe centre of the setting sun the user presses a shutter button tocapture the image. Unfortunately, when reviewing the image, the usernotices that a plastic carrier back was blown into shot over the sky andcaptured in the image. The presence of the carrier bag spoils the image.Image removal can identify features in the image, such as the plasticbag or lens flare for example, and using portions of the imagesurrounding the identified feature hide it from view. This may beachieved using any known technique. For example, an overlay may becreated having the same colours and textures of the image in the regionsurrounding the identified feature. When the overlay is placed over theplastic bag, it hides it.

Feature removal is a complex and daunting image editing feature for auser to use. A user typically has to make numerous actions to operateand adjust the feature removal process. It would be advantageous toprovide an easy to use and intuitive interface for applying a featureremoval process to an image.

FIGS. 17 a-d shows a first embodiment of a fourth aspect of theinvention. FIGS. 17 a to d shows an interface for interacting with afeature removal algorithm.

FIG. 17 a shows the image 1700 captured by the user of the boat passingin front of the sunset. The unsightly plastic bag 1705 is in the topleft corner. A first seagull 1710 and a second seagull 1715 are presentin the top right corner. A known feature detection/removal algorithm isused to identify to the processor features in the image that it is ableto hide. In this example, the feature detection/removal algorithm hasidentified the plastic bag 1705 and first and second seagulls 1710, 1715as features that it can hide.

The processor is configured to provide for the display of a userselectable marker 1720, 1725, 1730 to mark the position in the image1700 at which the identified features are present. Each selectablemarker is associated with a respective identified feature. Each of theidentified features is still visible beneath its associated marker. Tomake the visibility of the identified feature easier for the user, themarkers may be partially transparent or may be displayed and notdisplayed periodically. This will make it easier for the user to makeout which features in the image are hideable. FIG. 17 b shows the sameimage 1700 as FIG. 17 a with the markers displayed over the associatedidentified features. Marker 1720 marks the position of the feature 1705,marker 1725 marks the position of feature 1710, and marker 1730 marksthe position of feature 1715. The user selectable markers 1720, 1725,1730 are presented with a minus symbol to provide feedback to the userthat actuating the user selectable markers will hide the feature withwhich the marker is associated.

The user decides to remove the plastic bag 1705 from the image.Accordingly, processor receives a user selection of user selectablemarker 1720. In response to said user selection, the processor uses thefeature detection/removal algorithm or data previously receivedtherefrom to hide the plastic bag 1705. Accordingly, a modified versionof image 1700 is shown with the plastic bag 1705 hidden from view asshown in FIG. 17 c. The processor also changes the appearance of theselectable marker 1720 to present a plus symbol rather than a minussymbol. This provides convenient feedback to a user that the associatedidentified feature has been hidden and a further actuation of theselectable marker 1720 will show the identified feature 1705.

In FIG. 17 d, the processor has received a user selection of marker1730. Accordingly, the processor is configured to hide the secondseagull 1715 using the feature detection/removal algorithm. A furthermodified version of the image 1700 with the plastic bag 1705 and thesecond seagull 1715 hidden from view is displayed. As with marker 1720,the appearance of marker 1730 is altered to present a plus symbol ratherthan a minus symbol.

The user then decides that the image looked better with the secondseagull and provides user input to select the selectable marker 1730.The processor receives the user input and causes the second seagull 1715to be shown once again. Thus, the processor uses the featuredetection/removal algorithm or data previously received therefrom tomodify the image 1700 such that the second seagull is shown, while theplastic bag remains hidden. Thus, the user has returned to the positiondepicted in FIG. 17 c.

The processor may be configured to present an option 1735 to save animage incorporating the selected changes, if any, made by the featuredetection/removal algorithm.

FIGS. 18 a and 18 b show a second embodiment. In this embodiment theprocessor receives an image and uses the feature detection/removalalgorithm to identify features in the image that can be hidden fromview. The processor is configured to initially display a modifiedversion of the image with all of the identified features hidden by thefeature detection/removal algorithm. The user interface presented to theuser is similar in appearance and function to the previous embodimentbut in this instance user selectable markers 1820, 1825 and 1830 aredisplayed in a position corresponding to their associated features,which are hidden from view. The user selectable markers 1820, 1825 and1830 are now each presented with a plus symbol to provide feedback thatfeatures at those locations have been identified and hidden and can beshown upon user selection of the associated marker 1820, 1825 and 1830.

The user provides user input to select marker 1825 and 1830. Theprocessor receives the user input and causes the first seagull 1810 andsecond seagull 1815 to be shown. Thus, the processor uses the featuredetection/removal algorithm to generate an image that includes thefeatures 1810 and 1815 selected by the user. This modified image is thendisplayed as shown in FIG. 18 b. As in the previous embodiment, feedbackis provided by the processor by changing the symbol presented with themarkers 1825, 1830 to indicate that a further user selection will causethe processor to cause the marked features to be hidden once again. Anoption 1835 is also present to allow the user to save the image with theselected changes applied.

It will be appreciated that in the previous two embodiments the markersare presented as circles with a symbol therein. The circles may bedifferent sizes depending on the size of the identified feature. Themarker could be displayed as a line marking the perimeter of featurethat has been identified for addition or removal. The marker may not becentred on the feature it is marking. Instead, the marker could comprisean arrow that points to the position of the identified feature. It willbe appreciated symbols other than a plus or minus could be used, such aswords or colours or no symbols at all.

FIGS. 19 a to 19 b show a third embodiment which is similar to the firstembodiment of this aspect. In this embodiment the processor has receivedfrom the feature detection/removal algorithm the location of thefeatures that are hideable. However, rather than present the interfaceshown in FIG. 17 b, the processor is configured to perform a clusteringstep. The clustering step is performed to associate two or moreidentified features for treatment as a single identified feature.

The processor in this embodiment implements the clustering step bymeasuring the distance between the positions of features identified bythe feature detection/removal algorithm. Thus, the processor determinesa measurement point of the identified feature 1910, which may comprisesthe centre of the feature. Likewise, the processor determines ameasurement point of the identified feature 1915. The processor thenmeasures the distance between the two measurement points. If themeasured distance is less than a predetermined threshold, which could be100 pixels, then the processor clusters the two identified features1910, 1915. Thus, the processor creates a single user selectable marker1940 that it associates with both the identified feature 1910 and theidentified feature 1915. The marker 1940 is sized to surround both thefirst feature 1910 and second feature 1915 while being centred on amidpoint between the two clustered features 1910, 1915.

User input that selects the marker 1940, shown in FIG. 19 a, causes theprocessor to hide from view both the first seagull 1910 and secondseagull 1915. As discussed above, the processor may use data from thefeature detection/removal algorithm to hide the identified features orit may provide instructions for the feature detection/removal algorithmsuch that the algorithm generates a modified image with the identifiedfeatures 1910, 1915 removed as shown in FIG. 19 b.

FIGS. 20 a and 20 b show a fourth embodiment that operates insubstantially the same manner as the first embodiment of this aspect. Asshown in FIG. 20 a, the processor provides an interface that allows theselection of markers that mark hideable features identified by a featuredetection/removal algorithm. This is equivalent to the functionalitydescribed in relation to FIGS. 17 a and 17 b. However, on receipt ofuser input to select the selectable marker 2020, the processor hides thefeature 2005 but does not provide for display of the marker 2020 oncethe feature 2005 has been hidden. Thus, once the user has selected toremove the feature 2005, the action cannot be undone by way of a furtheractuation on a marker.

FIG. 21 shows a flow diagram illustrating the steps of marking theposition of at least one identified feature in an image which ishideable from view using a region of the image surrounding theidentified feature 2100, providing for display at least one selectablemarker that marks the position of at least one identified feature 2105,in response to user input selecting the marker hiding or showing theidentified feature 2110 and is self-explanatory.

Different types of image effect can be applied to an image. Typically,the image effect types have parameters associated therewith that mayadjust the strength of the image effect type. For example, one imageeffect type could be brightness and the parameter is brightness level.As a further example, contrast is a second image effect type and itsparameter is contrast level.

The making of adjustments to an image by applying image effects can be atime consuming process for a user as they navigate menu systems toselect the various image effect types and then select further options tochange the parameters associated with the image effect type.

It would be advantageous to provide a way of quickly and easilyadjusting image effect types and image effect parameters or any otherparameter.

In a first embodiment of this fifth aspect, a processor provides for thedisplay of an image that the user wishes to apply image effects to. Theprocessor is configured to allow adjustment of the parameters of twoeffect types simultaneously. The processor associates a first imageparameter with user swiping input along a first axis. A second imageparameter is associated with user swiping input along a second axis,different to the first axis. Typically, the first image parameter willbe associated with user swiping input relative to a first axis that isarranged vertically with respect to the display. Typically, the secondimage parameter will be associated with user swiping input relative to asecond axis that is arranged horizontally with respect to the display.

In the example of FIGS. 22 a to 22 b, the first image effect type isimage brightness, which is associated with the first, vertical axis. Thesecond image effect type is image contrast, which is associated with thesecond, horizontal axis.

In FIG. 22 a an image of a box 2205 is displayed. The user places theirfinger on the screen to begin a swiping input in the direction of arrow2210. Optionally, the processor displays a first indicator 2215 for thecurrent value of the first parameter and a second indicator 2220 for thecurrent value of the second parameter. Movement in the direction of thearrow 2210 results in movement relative to the first axis and movementrelative to the second axis.

FIG. 22 b shows the progressive user swiping input having reached apoint along the direction of the arrow 2210. In response to said userswiping input the processor is configured to determine the amount ofprogressive movement the user swiping input has made relative to thefirst axis and the amount of movement the user swiping input has maderelative to the second axis. Based on these amounts the processor isconfigured to simultaneously and progressively adjust both of therespective image parameters. The processor may achieve this in anyappropriate manner. For example, it may repeatedly project the userswiping input onto the first and second axis to track the movementrelative to those axes and update the image parameters accordingly.

In FIG. 22 b the progressive user swiping input has been measured withrespect to the first axis and second axis and the processor hasprogressively increased the first parameter by twenty increments and thesecond parameter by twelve increments. The processor is configured toprogressively display the two image effects applied to the image withtheir associated parameters during the user swiping input. This allowsthe user to see the effect of adjusting two image parameterssimultaneously.

FIG. 23 shows a flow diagram illustrating the steps of receivingprogressive user swiping input 2300, progressively adjusting a firstimage parameter in response to the amount of progressive user swipinginput relative to a first axis and simultaneously progressivelyadjusting a second image parameter, different to the first imageparameter, in response to the amount of progressive user swiping inputrelative to a second axis 2305; and providing for display of said imagewith modifications made thereto based on the first image parameter andsecond image parameter 2310 and is self-explanatory.

Image effects can be applied to live images. A live image is one that isdisplayed as it is received from a camera. It would be advantageous tobe able to adjust a parameter of an image effect as it is applied to alive image.

A first embodiment of a sixth aspect is shown in FIG. 24 a-c. In thisembodiment a live image is displayed from a camera. The processor isconfigured to receive user swiping input and in response to said swipinginput progressively adjust an image effect type or image parameterassociated with the image effect. In this embodiment the image effect isbrightness and the user swiping input is configured to adjust thebrightness level of the brightness effect applied to the live image.

Thus, in FIG. 24 a the user begins the swiping input at position 2410and the processor applies a first brightness level. By FIG. 24 b theprogressive user swiping reached point 2420 and has caused the processorto progressively apply a second brightness level stronger than the firstto the live image. Continuation of the user swiping input, shown in FIG.24 c at point 2430, causes the processor to progressively apply a thirdbrightness level strongest that the first and second levels to the liveimage.

Thus, the user is provided with an effective input method for adjustingimage parameters or image effect types applied to a live image.

FIG. 25 shows a flow diagram illustrating the steps of receivingprogressive user swiping input 2500, progressively adjusting a firstimage parameter of a live image in response to said progressive userswiping input 2505, and providing for display of said live image withprogressive modifications made thereto based on progressive adjustmentof the first image parameter 2510 and is self-explanatory.

FIG. 26 illustrates schematically a computer/processor readable medium2600 providing a program according to an example. In this example, thecomputer/processor readable medium is a disc such as a digital versatiledisc (DVD) or a compact disc (CD). In other examples, the computerreadable medium may be any medium that has been programmed in such a wayas to carry out an inventive function. The computer program code may bedistributed between the multiple memories of the same type, or multiplememories of a different type, such as ROM, RAM, flash, hard disk, solidstate, etc.

The processor of the above examples may form part of an apparatus whichmay be a portable electronic device, a laptop computer, a mobile phone,a Smartphone, a tablet computer, a personal digital assistant, a digitalcamera, a watch, a non-portable electronic device, a desktop computer, amonitor, a server, or a module/circuitry for one or more of the same.

Any mentioned apparatus/device and/or other features of particularmentioned apparatus/device may be provided by apparatus arranged suchthat they become configured to carry out the desired operations onlywhen enabled, e.g. switched on, or the like. In such cases, they may notnecessarily have the appropriate software loaded into the active memoryin the non-enabled (e.g. switched off state) and only load theappropriate software in the enabled (e.g. on state). The apparatus maycomprise hardware circuitry and/or firmware. The apparatus may comprisesoftware loaded onto memory. Such software/computer programs may berecorded on the same memory/processor/functional units and/or on one ormore memories/processors/functional units.

In some examples, a particular mentioned apparatus/device may bepre-programmed with the appropriate software to carry out desiredoperations, and wherein the appropriate software can be enabled for useby a user downloading a “key”, for example, to unlock/enable thesoftware and its associated functionality. Advantages associated withsuch examples can include a reduced requirement to download data whenfurther functionality is required for a device, and this can be usefulin examples where a device is perceived to have sufficient capacity tostore such pre-programmed software for functionality that may not beenabled by a user.

Any mentioned apparatus/circuitry/elements/processor may have otherfunctions in addition to the mentioned functions, and that thesefunctions may be performed by the sameapparatus/circuitry/elements/processor. One or more disclosed aspectsmay encompass the electronic distribution of associated computerprograms and computer programs (which may be source/transport encoded)recorded on an appropriate carrier (e.g. memory, signal).

Any “computer” described herein can comprise a collection of one or moreindividual processors/processing elements that may or may not be locatedon the same circuit board, or the same region/position of a circuitboard or even the same device. In some examples one or more of anymentioned processors may be distributed over a plurality of devices. Thesame or different processor/processing elements may perform one or morefunctions described herein.

With reference to any discussion of any mentioned computer and/orprocessor and memory (e.g. including ROM, CD-ROM etc), these maycomprise a computer processor, Application Specific Integrated Circuit(ASIC), field-programmable gate array (FPGA), and/or other hardwarecomponents that have been programmed in such a way to carry out theinventive function.

The applicant hereby discloses in isolation each individual featuredescribed herein and any combination of two or more such features, tothe extent that such features or combinations are capable of beingcarried out based on the present specification as a whole, in the lightof the common general knowledge of a person skilled in the art,irrespective of whether such features or combinations of features solveany problems disclosed herein, and without limitation to the scope ofthe claims. The applicant indicates that the disclosed aspects/examplesmay consist of any such individual feature or combination of features.In view of the foregoing description it will be evident to a personskilled in the art that various modifications may be made within thescope of the disclosure.

While there have been shown and described and pointed out fundamentalnovel features as applied to examples thereof, it will be understoodthat various omissions and substitutions and changes in the form anddetails of the devices and methods described may be made by thoseskilled in the art without departing from the scope of the disclosure.For example, it is expressly intended that all combinations of thoseelements and/or method steps which perform substantially the samefunction in substantially the same way to achieve the same results arewithin the scope of the disclosure. Moreover, it should be recognizedthat structures and/or elements and/or method steps shown and/ordescribed in connection with any disclosed form or examples may beincorporated in any other disclosed or described or suggested form orexample as a general matter of design choice. Furthermore, in the claimsmeans-plus-function clauses are intended to cover the structuresdescribed herein as performing the recited function and not onlystructural equivalents, but also equivalent structures. Thus although anail and a screw may not be structural equivalents in that a nailemploys a cylindrical surface to secure wooden parts together, whereas ascrew employs a helical surface, in the environment of fastening woodenparts, a nail and a screw may be equivalent structures.

1-20. (canceled)
 21. An apparatus comprising: at least one processor;and at least one memory including computer program code, the at leastone memory and the computer program code configured to, with the atleast one processor, cause the apparatus to perform at least thefollowing: in response to user input selecting a selectable marker, theselectable marker marking the position of at least one identifiedfeature in an image which is hideable from view using a region of theimage surrounding the identified feature, hide or show the identifiedfeature.
 22. The apparatus of claim 21, wherein the selectable markercomprises a ring or circular ring that surrounds the at least oneidentified feature.
 23. The apparatus of claim 21, wherein when aplurality of features are identified, each feature is marked by arespective selectable marker comprising a ring or circular ring thatsurrounds the respective feature.
 24. The apparatus of claim 21, whereinthe apparatus is configured to initially provide for display of theimage without any of the identified features hidden and the selectablemarkers are initially configured to allow for the hiding of a respectivefeature in response to said user input selection.
 25. The apparatus ofclaim 21, wherein the apparatus is configured to base on user input toselect a save command, save the image to memory with selected featureshidden and shown as they were selected on actuation of said savecommand.
 26. The apparatus of claim 21, wherein the selectable markersare displayed partially transparent.
 27. The apparatus of claim 21,wherein the selectable markers are periodically displayed and notdisplayed.
 28. The apparatus of claim 21, wherein the apparatus is aportable electronic device, a laptop computer, a mobile phone, aSmartphone, a tablet computer, a personal digital assistant, a digitalcamera, a watch, a server, or a module/circuitry for one or more of thesame.
 29. The apparatus of claim 21, wherein the apparatus is anon-portable electronic device, a desktop computer, a monitor, a server,or a module/circuitry for one or more of the same.
 30. The apparatus ofclaim 21, wherein the or each selectable markers includes a plus symbolwhen the identified feature is hidden in the image and includes a minussymbol when the identified feature is shown in the image.
 31. Theapparatus of claim 30, wherein the plus symbol and minus symbol areactuatable.
 32. The apparatus of claim 30, wherein the plus symbol for amarker allows for the hidden feature to be shown and the minus symbolfor a feature allows for the feature to be hidden.
 33. The apparatus ofclaim 21, wherein the apparatus is configured, when two or more featuresare identified, associate the two or more identified features such thatthey are treated as a single feature by determining if said two or moreidentified features satisfy a predefined clustering criterion.
 34. Theapparatus of claim 33, wherein the apparatus is configured such thatsaid association allows the two or more associated features to be shownor hidden together in response to said user input selection using asingle marker marking the position of the associated features.
 35. Theapparatus of claim 33, wherein the mark applied to mark the position ofassociated features comprises a ring that surrounds the associatedfeatures.
 36. The apparatus of claim 33, wherein the predefinedclustering criterion is satisfied if the region of the image surroundingthe one of the identified features overlaps with the region of the imagesurrounding the other of the identified features.
 37. The apparatus ofclaim 33, wherein the predefined clustering criterion is satisfied ifthe two or more identified features are within a predetermined distanceof one another.
 38. The apparatus of claim 33, wherein the predefinedclustering criterion is satisfied if the regions of the imagesurrounding the two or more identified features are of a hue that fallswithin a predetermined hue range of one another.
 39. A method, themethod comprising receiving user input selecting a selectable marker,the selectable marker marking the position of at least one identifiedfeature in an image which is hideable from view using a region of theimage surrounding the identified feature; and at least one of hiding andshowing the identified feature in response to said user input.
 40. Acomputer readable medium comprising computer program code storedthereon, the computer readable medium and computer program code beingconfigured to, when run on at least one processor, perform the method ofclaim 39.